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ABSTRACT 



Beginning in the fall of 1999, all programs of the Child and 
Youth Services Administration of the District of Columbia Commission on 
Mental Health Services were ordered to implement a performance improvement 
plan. This report looks at how successful one clinic was in improving 
performance in one year. The objectives were to: (1) increase the number of 

children receiving services; (2) increase the percentage of Hispanics and 
other multicultural groups receiving services,* (3) help children improve 
their school performance; (4) encourage families to participate,* and (5) 
monitor content and quality of services. The evaluation found that trying to 
improve all five objectives at the same time was a difficult task. The clinic 
had seen a marked increase in clients with histories of severe psychiatric 
problems, aggression, and violence. Public mental health systems with limited 
resources must make hard choices about resource allocation if the referrals 
continue to swell with complicated cases. The pressures to serve more 
clients, increase each family's involvement, and improve record keeping are 
often at cross-purposes. The recommendation was made that clear priorities be 
established so that the mission of prevention and training are maintained as 
performance improvement plans go forward. Data tables and the Service 
Satisfaction Questionnaire are appended. (Contains 10 references.) (JDM) 
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Abstract 



A public outpatient clinic implemented a 7-point performance improvement plan in FY 2000. 
Client census increased 10%. Hispanic clients increased to 15% of the total. Family satisfaction 
with the quality and appropriateness of care remained above 90%. Measures of family 
involvement in treatment planning showed no consistent trend. Quality of medical records 
documentation remained stable. Outcome measurement was not implemented. Family membership 
on the advisory board remained at one-third. This case study illustrates lessons for mental health 
institutions undertaking similar initiatives. 
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Meeting Performance Improvement Targets 
In a Children’s Outpatient Clinic 

How successfully can a public mental health clinic improve performance in one year? 
Nationwide, the public is demanding that providers improve accountability, increase productivity, 
serve a more diverse clientele, and collaborate better with clients and families in designing and 
delivering services (e g., Hatry, et al, 1997; Osher et al., 1999; D C. Action for Children, 1999). 
This paper is a case study of how one program, a children’s outpatient clinic, has striven to 
implement a performance improvement plan (AMBHA Committee on Quality Improvement and 
Clinical Services, 1998). 

Beginning in fall 1999, all programs of the Child and Youth Services Administration of the 
District of Columbia Commission on Mental Health Services were ordered to implement a seven- 
point performance improvement plan. The plan’s objectives were specified as follows: 

1. Increase the number of children receiving services, to serve up to 12% of District of 
Columbia children with serious emotional disturbance. 

2. Increase the percentage of Hispanic and other multicultural residents served to at least 
4%. 

3. Document that families are satisfied with the quality and appropriateness of care, using 
consumer satisfaction surveys. 

4. Encourage families to participate in the treatment planning process for their children. 

5. Monitor the content and quality of medical record documentation. 

6. Measure children’s improvement in school performance. 
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7. Increase family members’ participation on advisory boards to at least 51%. 

How are clinics to meet these challenges, particularly without additional resources? The 
plan would appear to require the program manager to guide significant changes in organizational 
culture and customary practices (e g., Kotter, 1996; Kouzes & Posner, 1995; Conner, 1995). For 
example, staff would need to find ways to serve more clients, while maintaining and even 
improving customer satisfaction and medical records documentation. They would need to find 
ways to serve clients from diverse cultural backgrounds, who often speak different languages. 
Relationships between staff and families would need to be rethought and renegotiated: 
hierarchical doctor-patient modes of relating would need to become more collaborative and 
family-driven (e.g., Osher et al, 1999). 

Results and Discussion 

Number of clients served. Over the period 10/99-7/00, the clinic’s census of registered 
clients was recorded quarterly. Results are shown in Table 1. From the first to the fourth quarter 
of plan implementation, census increased eight percent. Factors contributing to the growth 
included increased referrals from the community; use of more trainees (three social work students 
and an art therapy intern were placed at the clinic, in addition to the five psychology and two 
psychiatry trainees); somewhat higher caseloads for staff; and an effort to discharge more quickly 
clients who did not attend treatment sessions regularly. Impediments to further growth included a 
fluctuating flow of referrals; increasing severity of clients’ difficulties (necessitating more intensive 
services per client); the clinic’s ongoing commitment to prevention and training activities as well 



as treatment; and the existence of two to three vacancies in clinical staff positions for most of the 
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Multicultural clients served. The number of Hispanic and other multicultural clients served 



by the clinic increased from fifteen (eight percent of the total census) in the first quarter to thirty 
(14 percent) in the fourth quarter (Table 2). Crucial to this success was the establishment of a 
bilingual treatment team, comprising a clinical psychologist, a clinical social worker, and two 
trainees, one in each of these disciplines. The team conducted an outreach campaign in local 
community-based organizations and schools which generated a large number of referrals. 
Nevertheless, the initiative faced challenges by year’s end. The departure of a bilingual trainee 
would soon reduce the capacity of the team to treat Hispanic clients. There was ongoing difficulty 
in recruiting more bilingual staff to open positions. And a large variety of Asian, African, and 
other multicultural consumers continued not to be served at all. 

Family Satisfaction. The clinic had administered family satisfaction surveys for some years. 
A revised survey introduced a five-point Likert scale as well as open-ended questions 
(Satisfaction Survey is reproduced as Figure 1). The survey was also translated into Spanish. A 
ten percent sample of family members was given the survey during visits to the clinic. Average 
satisfaction was at 4.8 on a five-point scale in the March, 2000 survey. Challenges to 
implementation included the low response rate to surveys, and the difficulty in selecting a random 
sample. Moreover, the nearly uniformly positive responses on the survey meant that few if any 
suggestions were generated for program improvement. 

Family Involvement in Treatment Planning. This was measured by a retrospective review 
of six items documented in the medical record. Families were considered involved if they: 
discussed the treatment plan with the clinician; attended the treatment planning conference; signed 
the treatment plan (preferably on the date of the treatment planning conference); and/or indicated 
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their approval of the treatment plan. Percent compliance with indicators fluctuated over the 
course of the year, (see Table 3) with the first quarter at 67% and the fourth quarter at 60%. 
Fluctuation appeared to be due largely to sampling variation, and to failure of some clinicians to 
document all treatment-planning-related contacts with families. More important, significant 
challenges to clinician-family collaboration are posed by the multi-problem nature of many client 
families, and the fact that working parents cannot often take time off from work to attend clinic 
sessions. 

Medical records quality. The quality of medical records in the clinic was measured by 
rating randomly selected records on ten items (Table 4). Average compliance on all items ranged 
between 78 and 87 percent, with no discernible trend over the year. When sorted by clinician, 
these data suggested that whereas certain staff maintained consistently high rates of compliance, 
others continued habits of substandard record keeping. It also appeared that some clinicians were 
affected more than others by staff vacancies and assumed additional job duties, compromising 
their record keeping. To rectify this situation, the following measures have begun to be 
implemented: increased frequency and number of records reviews, utilizing reviewers from other 
programs; more written feedback to individual clinicians; and a more explicit link between record 
keeping and annual performance ratings. 

Children’s school performance. Children’s school performance was envisioned as a set of 
outcome measures, and discussed extensively in the planning phases of the performance 
improvement plan. Outpatient clinics were exempted from implementing these measures, for 
several reasons. Staff resources were insufficient to gather report card data from schools on a 
regular basis. No method was devised to aggregate the multiple data from a report card into 
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meaningful summary measures for individual clients. Finally, there were no resources available to 
analyze the data, had it been gathered. Individual clinicians continue to track their clients’ school 
performance on a case-by-case basis. 

i 

Advisory boards will have at least 51% consumer membership. Parents of current and 
former clients constituted two of six members of the clinic’s advisory board. The center director 
frequently asked staff and current board members to suggest additional appropriate candidates for 
board membership, and no suggestions were forthcoming. Those family members asked directly 
have declined to participated, citing work and family commitments. It appears that recruiting 
additional family members to the advisory board, and/or devising other ways for family members 
to participate in program evaluation and governance, is a long-term project (Woodbridge & 
Huang, 2000). 

Conclusions 

There is an upper limit on the number of clients a clinic can serve. The clinic has seen a 
marked increase during the past year in clients with histories of multiple hospitalizations, severe 
psychiatric symptoms, criminal history, aggression and violence, and history of residential 

f- 

treatment. The more severely-disturbed the clientele which is deemed suitable for outpatient 
treatment, the more service time each client will require, and therefore the smaller the caseload 
each clinician, and the clinic as a whole, ought to carry (Henggeler et al, 1998). If the clinic 
receives more of these referrals and cannot refer them for services elsewhere, then the overall 
clinic target for the number of clients served will need to be decreased. Public mental health 
systems with limited resources therefore face hard choices in resource allocation. 

Making a children’s mental health system truly “family-driven” takes time. It sometimes 
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seems that families have little time to attend meetings, fill out surveys, or join advisory boards. On 
the other hand, staff and administrators need to be more flexible, accessible and available, to set 
askid time to reach out to families, to really invite families and really listen to them. In addition, 
other means of increasing families’ involvement will need to be devised. These may include public 
meetings, advisory councils, participation in administrative decisions, and use of hired parent 
advocates. Such efforts, it is anticipated, will bear rich fruit in increased communication, 
understanding, and perceived responsiveness. 

Measuring outcomes requires an investment of organizational resources. The data needs 
of administration, the public, and clinicians themselves are not identical. Staff tend to perceive 
data collection as both burdensome and irrelevant to clinical decision-making (Bickman, 1999). 

It is hard to improve everything at once. The pressures to serve more clients, increase each 
family’s involvement, and improve record keeping are often at cross purposes with one another. 
These priorities in turn draw time and attention away from other core activities of the clinic’s 
mission such as prevention and training. These issues will need to be addressed, and clear 
priorities established, as implementation of the performance improvement plan goes forward in 
succeeding years. 
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Tables 



Table 1 



Client Census by Quarter 



Quarter Client Census 



10/99 


194 


1/00 


215 


4/00 


218 


7/00 


210 
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Table 2 



Multicultural Consumers Served 



* 

Quarter Multicultural Consumers Served Percent of Clinic Census 



10/99 


15 


8 


1/00 


25 


12 


4/00 


33 


15 


7/00 


30 


14 
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Figure 1 



Northwest Family Center 
1536 U Street. N.W., Third Floor 
Washington. D.C. 20009 

. SERVICE SAT I SFACTION OtTESTTONNATRK 



Dear parent /guard i an : 



The Northwest Family Center would like your help in improving our 

complete t t h ° Children 8nd lilies. Please take a few^inutls 
complete this survey. Your name will not be used in any reporting 

?lmily% S nro e t y hers Y ° Ur SU88eSti °" S wi 1 1 hel *> to better serve your 



When you have finished, please put your survey in the box provided. 

5 , ffr-X 

David B. Sacks, Psy.D. 

Clinical Administrator 



How satisfied are you that ... Refer Seldoi So.cti.es often Always 

1 2 ) 4 5 

1. Northwest Family Center staff 
treat you and your family with 

dignity and respect. 123 is 



2. Staff is honest with you and 

your family. 1 j 

3. Staff listens to you and your 

family’s problems. ] j 

4. You and your family can freely 

ask questions about the treat- 
ment plan. 1 j 

5. Staff clearly answers your and your 

family’s questions about the treat- 
ment plan. I 2 

6. The staff clearly implement your 

family’s requests for specific 
services/treatment interventions. 1 2 

Z-L The s t a f_f_make.s— a— spe e-i-a-1 — e-f-f-or t 

to include you and your family 

members . i , 
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How satisfied are you that ... Mever Seldoi Joietiiei Often Alny« 

1 2 3 4 5 



8. You and your family are active 
participants in the treatment 

plan. 12 

9. You and your family are expected 

to attend treatment planning 
meetings. 1 j 

10. The activities included in the 
treatment plan will help to 

improve the problem. l 2 

11. The finished treatment plan clearly 

includes you and your family’s 
concerns . 12 

12. You and your family understand 

the treatment plan. l 2 



3 



3 



3 



3 



3 



4 5 



4 5 



4 5 



4 5 



4 5 



13. You and your family are well 
treated by the staff. 



5 



What do you find most helpful about your work with Northwest Familv 
Center? 



Finally, please write your suggestions for improving our services: 



If you would like to be contacted regarding this survey, please 
given your name and telephone number, and the Clinical Administra- 
tor will call you: 



03/00 
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